Skip to content

[pr] Removing conflicting crates#1

Merged
DudessaPr merged 13 commits intofeat/runtimefrom
feat/zstd-remove
Jul 9, 2024
Merged

[pr] Removing conflicting crates#1
DudessaPr merged 13 commits intofeat/runtimefrom
feat/zstd-remove

Conversation

@DudessaPr
Copy link
Copy Markdown

@DudessaPr DudessaPr commented Jul 8, 2024

Problem

Cannot compile riscv in svm rollup because of zstd, lz4 and bzip2
Also had issues with solana-perf, accountsdb and symlink in solana-runtime

Summary of Changes

Removed zstd, lz4 and bzip2 from runtime, account decoder and accounts db
Removed solana-perf
Fixed accountsdb for solana-runtime
Removed symlink from solana-runtime

Fixes #

Copy link
Copy Markdown

@neutrinoks neutrinoks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

/// The maximum supported offset for hot accounts storage.
const MAX_HOT_ACCOUNT_OFFSET: usize = u32::MAX as usize * HOT_ACCOUNT_ALIGNMENT;
// const MAX_HOT_ACCOUNT_OFFSET: usize = u32::MAX as usize * HOT_ACCOUNT_ALIGNMENT;
const MAX_HOT_ACCOUNT_OFFSET: usize = (u32::MAX as u64 * HOT_ACCOUNT_ALIGNMENT as u64) as usize;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why you need to change this here?

Copy link
Copy Markdown
Author

@DudessaPr DudessaPr Jul 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got an error about it, something related to overflow of usize

sov-prover-guest-celestia:    Compiling spl-token-metadata-interface v0.3.3
sov-prover-guest-celestia:    Compiling spl-transfer-hook-interface v0.6.3
sov-prover-guest-celestia:    Compiling spl-token-2022 v3.0.2
sov-prover-guest-celestia: error[E0080]: evaluation of constant value failed
sov-prover-guest-celestia:   --> /Users/dulatrakhymkul/.cargo/git/checkouts/agave-63bb32cf49464624/0b7ae2f/accounts-db/src/tiered_storage/hot.rs:72:39
sov-prover-guest-celestia:    |
sov-prover-guest-celestia: 72 | const MAX_HOT_ACCOUNT_OFFSET: usize = u32::MAX as usize * HOT_ACCOUNT_ALIGNMENT;
sov-prover-guest-celestia:    |                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ attempt to compute `usize::MAX * 8_usize`, which would overflow
sov-prover-guest-celestia: 
sov-prover-guest-celestia: note: erroneous constant encountered
sov-prover-guest-celestia:    --> /Users/dulatrakhymkul/.cargo/git/checkouts/agave-63bb32cf49464624/0b7ae2f/accounts-db/src/tiered_storage/hot.rs:119:21
sov-prover-guest-celestia:     |
sov-prover-guest-celestia: 119 |         if offset > MAX_HOT_ACCOUNT_OFFSET {
sov-prover-guest-celestia:     |                     ^^^^^^^^^^^^^^^^^^^^^^
sov-prover-guest-celestia: 
sov-prover-guest-celestia: note: erroneous constant encountered
sov-prover-guest-celestia:    --> /Users/dulatrakhymkul/.cargo/git/checkouts/agave-63bb32cf49464624/0b7ae2f/accounts-db/src/tiered_storage/hot.rs:122:17
sov-prover-guest-celestia:     |
sov-prover-guest-celestia: 122 |                 MAX_HOT_ACCOUNT_OFFSET,
sov-prover-guest-celestia:     |                 ^^^^^^^^^^^^^^^^^^^^^^
sov-prover-guest-celestia: 
sov-prover-guest-celestia: For more information about this error, try `rustc --explain E0080`.
sov-prover-guest-celestia: error: could not compile `solana-accounts-db` (lib) due to 1 previous error
sov-prover-guest-celestia: warning: build failed, waiting for other jobs to finish...
error: failed to run custom build command for `risc0 v0.3.0 (/Users/dulatrakhymkul/Documents/GitHub/svm-rollup/crates/rollup/provers/risc0)`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiplying u32::MAX as usize by 8 should work! The compiler does not detect that you don't want usize::MAX. Maybe try: (u32::MAX as usize) * HOT_ACCOUNT_ALIGNMENT...

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DudessaPr did you end up trying Michael's suggestion here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it did not work
For some reason compiler complains
As long as it works correctly I think it is better to leave it like that

Comment thread runtime/Cargo.toml
@DudessaPr DudessaPr changed the title Removing conflicting compression crates [pr] Removing conflicting crates Jul 9, 2024
Copy link
Copy Markdown

@neutrinoks neutrinoks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment about the conversion issue 👍

/// The maximum supported offset for hot accounts storage.
const MAX_HOT_ACCOUNT_OFFSET: usize = u32::MAX as usize * HOT_ACCOUNT_ALIGNMENT;
// const MAX_HOT_ACCOUNT_OFFSET: usize = u32::MAX as usize * HOT_ACCOUNT_ALIGNMENT;
const MAX_HOT_ACCOUNT_OFFSET: usize = (u32::MAX as u64 * HOT_ACCOUNT_ALIGNMENT as u64) as usize;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiplying u32::MAX as usize by 8 should work! The compiler does not detect that you don't want usize::MAX. Maybe try: (u32::MAX as usize) * HOT_ACCOUNT_ALIGNMENT...

@DudessaPr DudessaPr merged commit 05fc91e into feat/runtime Jul 9, 2024
DudessaPr added a commit that referenced this pull request Jul 10, 2024
* remove ztsd

* remove lz4

* remove gz and fix others

* remove filetime/tar

* remove perf from runtime

* fix accountsdb const

* comment out solana perf

* remove symlink

* update cargo lock

* remove flate2 again

* retrun bz instead of hz

* try brackets

* revert

return removed and comment
DudessaPr added a commit that referenced this pull request Jul 10, 2024
* remove ztsd

* remove lz4

* remove gz and fix others

* remove filetime/tar

* remove perf from runtime

* fix accountsdb const

* comment out solana perf

* remove symlink

* update cargo lock

* remove flate2 again

* retrun bz instead of hz

* try brackets

* revert

return removed and comment
neutrinoks pushed a commit that referenced this pull request Jul 17, 2024
* remove ztsd

* remove lz4

* remove gz and fix others

* remove filetime/tar

* remove perf from runtime

* fix accountsdb const

* comment out solana perf

* remove symlink

* update cargo lock

* remove flate2 again

* retrun bz instead of hz

* try brackets

* revert

return removed and comment
petarvujovic98 pushed a commit that referenced this pull request Oct 24, 2025
* Apply the retry code to the async pubsub client

Create a test server

```ts
import http from "http";

import { WebSocketServer } from "ws";

let attemptCount = 0;

const server = http.createServer();
const wss = new WebSocketServer({ noServer: true });

wss.on("connection", (ws) => {
  ws.send("Connection accepted.");
  ws.on("message", (msg) => console.log(`Received: ${msg}`));
});

server.on("upgrade", (req, socket, head) => {
  attemptCount += 1;

  if (attemptCount <= 4) {
    socket.write("HTTP/1.1 429 Too Many Requests\r\n\r\n");
    socket.destroy();
    console.log(`Rejected connection #${attemptCount} (429)`);
    return;
  }

  wss.handleUpgrade(req, socket, head, (ws) => {
    wss.emit("connection", ws, req);
    console.log("Connection accepted on attempt", attemptCount);
  });
});

server.listen(8080, () => {
  console.log("Server listening on port 8080");
});
```

Run `test_slot_subscription_async`:

```
Rejected connection #1 (429)
Rejected connection #2 (429)
Rejected connection #3 (429)
Rejected connection #4 (429)
Connection accepted on attempt 5
Received: {"id":1,"jsonrpc":"2.0","method":"slotSubscribe","params":[]}
```

* `s/async_with_retry/with_retry/`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants